System, method, and computer-readable medium for displaying virtual image based on position detected by sensor

ABSTRACT

A system for displaying an image of a virtual space on a display obtains detection data from position detection sensors, the position detection sensors including a position detection sensor attached to a user. The system calculates a distance between a pair of the position detection sensors based on the detection data, and sets, based on the distance, a boundary for determining a motion of the user. The system determines, when the motion of the user occurs, whether a positional relationship between one of the position detection sensors and the boundary satisfies a condition, and, responsive to the positional relationship satisfying the condition, executes an action corresponding to the motion of the user in the virtual space.

This application claims the benefit of priority from Japanese PatentApplication No. 2018-154384 filed Aug. 21, 2018, the entire contents ofthe prior application being incorporated herein by reference.

BACKGROUND 1. Field

The following description relates to a system, a method, and acomputer-readable medium for displaying an image.

2. Description of Related Art

There is a known game in which the motions of a user in a real space aredetected using various types of sensors or the like and reflected onobjects in a virtual space. Japanese Laid-Open Patent Publication No.10-214155 discloses an input device used for a fishing game. The inputdevice is equipped with a sensor capable of detecting an accelerationand inclination and connected to a game processing device. With atrigger button turned on, when the input device is moved by a motionlike an actual motion of casting a fishing rod, an image in which thefishing rod is being cast in a water surface is displayed on a display,which is connected to the game processing device.

In the above-described system, when the input device is moved a littlewith the trigger button turned on, the image in which the fishing rod isbeing cast in the water surface may be displayed against the user'sintention. When the motion of the user in the real world deviates fromthe reflected image in such a manner, the user may experiencediscomfort.

SUMMARY

It is an objective of the present disclosure to display an image inwhich a user experiences less discomfort when a motion of the user isreflected on an action in a virtual space.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

In one general aspect, a system for displaying an image of a virtualspace on a display is provided. The system includes circuitry configuredto obtain detection data from a plurality of position detection sensorsincluding a position detection sensor attached to a user, calculate adistance between a first position detection sensor and a second positiondetection sensor of the position detection sensors based on thedetection data, set, based on the distance, a boundary for determining amotion of the user, determine, when the motion of the user occurs,whether a positional relationship between one of the position detectionsensors and the boundary satisfies a condition, and, responsive to thepositional relationship satisfying the condition, execute an actioncorresponding to the motion of the user in the virtual space.

In another general aspect, a method for displaying an image of a virtualspace on a display using a computer including circuitry is provided. Themethod includes obtaining detection data from position detectionsensors, the position detection sensors including a position detectionsensor attached to a user, calculating a distance between a firstposition detection sensor and a second position detection sensor of theposition detection sensors based on the detection data, setting, basedon the distance, a boundary for determining a motion of the user,determining, when the motion of the user occurs, whether a positionalrelationship between one of the position detection sensors and theboundary satisfies a condition, and, responsive to the positionalrelationship satisfying the condition, executing an action correspondingto the motion of the user in the virtual space.

In a further general aspect, a non-transitory computer-readable mediumthat stores a program is provided. The program, when executed by acomputer including circuitry, causes the circuitry to obtain detectiondata from position detection sensors, the position detection sensorsincluding a position detection sensor attached to a user, calculate adistance between a first position detection sensor and a second positiondetection sensor of the position detection sensors based on thedetection data, set, based on the distance, a boundary for determining amotion of the user, determine, when the motion of the user occurs,whether a positional relationship between one of the position detectionsensors and the boundary satisfies a condition, and, responsive to thepositional relationship satisfying the condition, execute an actioncorresponding to the motion of the user in the virtual space.

Other features and aspects will be apparent from the following detaileddescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram schematically showing a system of displaying animage according to a first embodiment.

FIG. 2 is a block diagram showing the system of FIG. 1.

FIG. 3 is a diagram illustrating a method for setting a boundary in thefirst embodiment.

FIG. 4 is a diagram illustrating how the boundary is used in the firstembodiment.

FIG. 5 is a diagram illustrating how the boundary is used in the firstembodiment.

FIG. 6A is a table illustrating data used to calculate the movementdistance of an object in the first embodiment.

FIG. 6B is a map illustrating data used to calculate the movementdistance of an object in the first embodiment.

FIG. 7 is an exemplary view showing a game view in the first embodiment.

FIG. 8 is a flowchart illustrating a procedure of setting the boundaryin the first embodiment.

FIG. 9 is a flowchart illustrating the progress of the game in the firstembodiment.

FIG. 10A is a diagram showing movement of the object toward the rearwhen the boundary is used according to a second embodiment.

FIG. 10B is a diagram showing movement of the object toward the rearright when the boundary is used in the second embodiment.

FIG. 10C is a diagram showing movement of the object toward the rearleft when the boundary is used in the second embodiment.

FIG. 11A is a diagram showing movement of the object toward the frontwhen a boundary is used in a system of displaying an image according tothe second embodiment.

FIG. 11B is a diagram showing movement of the object toward the frontleft when the boundary is used in the system according to the secondembodiment.

FIG. 11C is a diagram showing movement of the object toward the frontright when the boundary is used in the system according to the secondembodiment.

FIG. 12 is a diagram showing an upward movement of the object when theboundary is used in the second embodiment.

FIG. 13 is a flowchart illustrating the progress of the game in thesecond embodiment.

FIG. 14A is a diagram illustrating the setting of a boundary when theboundary is used in a system of displaying an image according to amodification.

FIG. 14B is a diagram illustrating the determination of whether a motionhas started when the boundary is used in the system according to themodification.

FIG. 14C is a diagram illustrating the determination that the motion iscomplete when the boundary is used in the system according to themodification.

FIG. 15A is a diagram illustrating the setting of a boundary when theboundary is used in a system of displaying an image according to afurther modification.

FIG. 15B is a diagram illustrating the determination of the type ofaction when an ankle position is located below the boundary when theboundary is used in the system according to the modification.

FIG. 15C is a diagram illustrating the determination of the type ofaction when the ankle position is located above the boundary when theboundary is used in the system according to the modification.

Throughout the drawings and the detailed description, the same referencenumerals refer to the same elements. The drawings may not be to scale,and the relative size, proportions, and depiction of elements in thedrawings may be exaggerated for clarity, illustration, and convenience.

DETAILED DESCRIPTION

This description provides a comprehensive understanding of the methods,apparatuses, and/or systems described. Modifications and equivalents ofthe methods, apparatuses, and/or systems described are apparent to oneof ordinary skill in the art. Sequences of operations are exemplary, andmay be changed as apparent to one of ordinary skill in the art, with theexception of operations necessarily occurring in a certain order.Descriptions of functions and constructions that are well known to oneof ordinary skill in the art may be omitted.

Exemplary embodiments may have different forms, and are not limited tothe examples described. However, the examples described are thorough andcomplete, and convey the full scope of the disclosure to one of ordinaryskill in the art.

First Embodiment

An image processing system according to a first embodiment will now bedescribed.

As shown in FIG. 1, the image processing system includes a housing 11, atracking system 20, a head-mounted display (HMD) 30, and a gameprocessing device 50. The tracking system 20 includes one or moretracking assistance devices 21 and includes tracking sensors 22. Thegame processing device 50 corresponds to a detection data obtainingunit, a distance calculation unit, a setting unit, a determination unit,and an action execution unit. The tracking sensors 22 correspond toposition detection sensors. The game processing device 50 may include adedicated hardware circuit (for example, application specific integratedcircuit: ASIC) that executes hardware processing on at least part of theprocesses executed by itself. That is, the game processing device 50 maybe circuitry including one or more processors that operate according toa computer program (software), one or more dedicated hardware circuitsthat execute at least part of various processes, or a combinationthereof.

The housing 11 imitates a swing. The housing 11 includes a support 12and a seat 15, which is hung by the support 12 via a hanging portion 13.The seat 15 includes a seating surface and a rear surface, which islocated on the side opposite from the seating surface. The rear surfaceof the seat 15 is provided with a tracking sensor 22. Further, theswinging range of the seat 15 is limited to a predetermined range suchthat the seat 15 does not swing greatly. For example, the hangingportion 13 or the seat 15 may be provided with a member that limits theamplitude of swing. Alternatively, the hanging portion 13 or the seat 15may have a larger weight.

The HMD 30 is attached to the head of a user 200. The user 200 is seatedon the seat 15. The tracking sensor 22 is attached around the ankle ofthe user 200. The position of the tracking sensor 22 does not have to bearound the ankle and may be, for example, the toe or heel. The trackingsensor 22 may itself be attachable to the body of the user 200.Alternatively, the tracking sensor 22 may be attachable to the user 200by, for example, an attachment member such as a belt.

The game processing device 50 is located in the space where the housing11 is arranged or located at other positions. The game processing device50 is connected to the HMD 30 via a communication cable or in a wirelessmanner, thereby allowing data to be transmitted and received in abilateral direction. The game processing device 50 performs a processfor displaying an image of the virtual space on the HMD 30 and causingthe game to progress.

The tracking system 20 uses the tracking sensors 22 and sensors arrangedin a tracking space. The tracking assistance devices 21 are located inthe space where the housing 11 is arranged. The tracking assistancedevices 21 and the HMD 30 cooperate to transmit, to the game processingdevice 50, detection data for detecting the position of the HMD 30.Further, the tracking assistance devices 21 and the tracking sensors 22cooperate to transmit, to the game processing device 50, detection datafor detecting the positions of the tracking sensors 22.

The game processing device 50 detects the motion of a foot of the user200 seated on the seat 15 based on the data transmitted from at leastone of the tracking assistance device 21 and the tracking sensor 22. Thegame processing device 50 detects the user's motion of, for example,moving both feet in synchronization or moving one foot at a time todetermine whether the detected motion satisfies a predeterminedcondition. When determining that the detected motion satisfies thepredetermined condition, the game processing device 50 executes, in thevirtual space, a virtual action of a virtual character (avatar orcharacter) corresponding to the user. In the present embodiment,description will be made of a game in which the user 200 performs amotion of kicking forward in the real world so that the virtualcharacter corresponding to the user 200 executes an action of kickingaway a shoe object worn on the foot frontward in the virtual space,which is referred to as a kick-the-shoe-on-the-swing motion.

The HMD 30, the tracking sensor 22, and the tracking assistance device21 will now be described in detail with reference to FIG. 2.

The HMD 30 is a wearable computer. The HMD 30 is a non-transparentdisplay including a housing that covers both eyes or is a transparentdisplay. The non-transparent display may display, on one or moredisplays, an image to be visually recognized by the left eye and animage to be visually recognized by the right eye. Alternatively, thenon-transparent display may display, on a display, an image common tothe left eye and the right eye. The transparent display may display, ona display, an image captured by a camera provided in a head-mounteddisplay or another device. The display may be formed by a half mirror ormade of a transparent material so that the display allows the real worldto be visually recognized. The HMD 30 may be a display fixed to, forexample, a housing or a frame. Alternatively, the HMD 30 may be amulti-functional telephone terminal such as a smartphone fixed to apredetermined housing in a removable manner. The HMD 30 displays animage of, for example, virtual reality (VR), augmented reality (AR),which provides content of the virtual space while causing the user tovisually recognize the real world, or mixed reality (MR), whichencompasses virtual and augmented reality. In the present embodiment,the HMD 30 will be described as a non-transparent display.

The HMD 30 includes an information processor 31, a measurement device32, and a display 33. The information processor 31 executes a processfor causing the game to progress. The information processor 31 is notlimited to one that performs software processing on all processesexecuted by itself. For example, the information processor 31 mayinclude a dedicated hardware circuit (for example, application specificintegrated circuit: ASIC) that executes hardware processing on at leastpart of the processes executed by itself. That is, the informationprocessor 31 may be circuitry including one or more processors thatoperate according to a computer program (software), one or morededicated hardware circuits that execute at least part of variousprocesses, or a combination thereof.

The processor includes a calculation processor such as CPU, MPU, or aGPU and includes a storage medium such as RAM or ROM. Further, theinformation processor 31 includes a storage medium (memory), which isstorage such as hard disk drive (HDD) or solid state drive (SSD). Atleast one of these storage media stores program codes or instructionsconfigured to cause the CPU to execute processes. The storage media, orcomputer readable media, include any type of media that are accessibleby general-purpose computers and dedicated computers. The measurementdevice 32 is a device that detects at least the orientation of the HMD30 and is, for example, an inertial measurement unit (IMU). The inertialmeasurement unit includes, for example, a gyro sensor and anacceleration sensor and detects at least one of the rotational angle,angular angle, and acceleration with respect to an X-axis, Y-axis, andZ-axis. The display 33 is, for example, an organic EL display or aliquid crystal display.

The tracking sensor 22 detects the position and orientation of thetracking sensor 22. The tracking sensor 22 includes, for example, aninertial measurement unit in the same manner as the HMD 30.

The tracking sensor 22 cooperates with multiple tracking assistancedevices 21 to output signals corresponding to the position andorientation of the tracking sensor 22. The tracking sensor 22 and thetracking assistance devices 21 cooperate to detect the position of thetracking sensor 22 in the space. One example of the tracking assistancedevices 21 is a multi-axis laser oscillator. In this case, the trackingassistance devices 21 are arranged diagonally on the upper side of thespace where the housing 11 is arranged. The tracking assistance devices21 emit pulse laser light. The tracking sensor 22 includes a sensor thatdetects laser light and detects the position and orientation of thetracking sensor 22 while performing synchronization usingsynchronization pulses. For example, Vive Tracker (registered trademark)and Vive Base Station (registered trademark), which are offered by HTCCorporation, can be used for the tracking sensor 22 and the trackingassistance device 21.

The game processing device 50 includes a position locator 51, an imageprocessor 52, a progress manager 53, and a data memory 60. The positionlocator 51 determines the orientation of the HMD 30 based on the dataobtained from at least one of the HMD 30 and the tracking assistancedevice 21. In addition to the orientation of the HMD 30, the positionlocator 51 may determine the position of the HMD 30. Further, theposition locator 51 determines the orientation and position of thetracking sensor 22 based on the data obtained from at least one of thetracking sensor 22 and the tracking assistance device 21.

The image processor 52 displays an image of the virtual space on thedisplay 33 in accordance with the orientation and position of the HMD 30determined by the position locator 51. Further, the image processor 52displays, as part of the virtual space, part of the body of the virtualcharacter corresponding to the user.

The progress manager 53 manages the progress of the game and performsprocesses other than the ones executed by the position locator 51 andthe image processor 52. The progress manager 53 starts the game when agame start condition is satisfied. For example, the game start conditionis that the user or the manager of the game inputs an instruction.Further, the progress manager 53 ends the game when a game end conditionis satisfied. The game end condition can be changed depending on thegame. For example, the game end condition is that an elapsed time fromthe beginning of the game has reached a time limit, a score of the userhas reached a threshold value, or a mission has been achieved.

The data memory 60 is a storage medium such as HDD or SSD. The datamemory 60 stores a game processing program and other programs. Theposition locator 51, the image processor 52, and the progress manager 53execute computer-readable instructions described in these programs forthe game to progress.

The data memory 60 stores setting information 61, game space data 62,and game progress data 63. The setting information 61 includes boundaryinformation, which is set based on the position of the tracking sensor22.

The game space data 62 is data for drawing a game-playing space. Forexample, the game space data 62 includes data for drawing the backgroundof a game field or data for drawing an object in the virtual space. Suchan object includes an object such as an enemy that moves in the gamefield and an object displayed only when a predetermined condition issatisfied. The game space data 62 includes position information of suchan object in the virtual space.

The game progress data 63 is used to manage the progress of the game andupdated in accordance with the progress of the game. The game progressdata 63 includes user attribute information, which is associated withthe user. The user attribute information includes at least one of usersetting information, user parameters, game media (or game content), andattribute information associated with the game media. User settinginformation is, for example, the gender, age or age bracket, location,or email address of the user. A user parameter is, for example, aparameter of the virtual character corresponding to the user. A userparameter is, for example, a game result such as a score, the number oftimes the user has won, or the number of times the user has lost.Alternatively, a user parameter is, for example, a battle parameter suchas level, status, competency, skill, ability, attack ability, defenseability, hit point (HP), life, health value, restoration ability, spell,or job. Game media are electronic data (content) used in the game. Gamemedia may be, for example, obtained, possessed, used, managed,exchanged, synthesized, reinforced, sold, abandoned, or donated by theuser in the game. Game media include, for example, any medium such as acard, an item, a virtual currency, a ticket, a character, and an avatar.Attribute information of game media is, for example, a parameter of agame medium and is the same as user attribute information. Instead ofgame media, user parameters or attribute information associated withgame media may be, for example, obtained, possessed, used, managed,exchanged, synthesized, reinforced, sold, abandoned, or donated.Further, the game progress data 63 may include enemy-relatedinformation. Enemy-related information includes, for example, how easilyan enemy can be beaten such as the health value of the enemy.

An input operation device 65 is connected to the game processing device50 such that data can be transmitted and received. The input operationdevice 65 is a pointing device such as a mouse, keyboard, or a touchpanel unit. The input operation device 65 is connected to the gameprocessing device 50 via a communication cable or in a wireless manner.The input operation device 65 is used by the user or the game manager.

The method for setting a boundary for determining an action of the user200 will now be described with reference to FIG. 3. The user 200 isseated on the seat 15 with the tracking sensor 22 attached around eachankle. FIG. 3 shows only the right foot. The bottom surfaces of the feet(shoes) of the user 200 do not necessarily have to be in contact with afloor surface 500.

The game processing device 50 obtains detection data from the trackingsensors 22 to determine the positions of the tracking sensors 22. Thisallows the game processing device 50 to obtain the position of thetracking sensor 22 attached to the seat 15, the position of the trackingsensor 22 attached to the left foot, and the position of the trackingsensor 22 attached to the right foot. The position of the trackingsensor 22 attached to the seat 15, the position of the tracking sensor22 attached to the left foot (not shown in FIG. 3), and the position ofthe tracking sensor 22 attached to the right foot are respectivelyreferred to as a seating position P1 (X1, Y1, Z1), a left foot positionP2 (X2, Y2, Z2), and a right foot position P3 (X3, Y3, Z3).

The direction which is parallel to the floor surface 500 and in whichthe face of the user 200 seated on the seat 15 is oriented is referredto an X-direction (front-rear direction). The direction that is parallelto the floor surface 500 and is orthogonal to the X-direction isreferred to as a Z-direction (left-right direction). The normaldirection of the floor surface 500 is referred to as a Y-direction(vertical direction). Instead of these directions, any coordinate systemcan be set for the game processing device 50.

The game processing device 50 calculates a distance Ly between theseating position P1 and the left foot position P2 in the Y-direction ora distance Ly between the seating position P1 and the right footposition P3 in the Y-direction. Instead, the distances Ly may both becalculated. This is used to measure the length of the lower leg of theuser 200 from the knee to the ankle. Alternatively, for example, thedistance from the Y-coordinate of the seating position P1 to theintermediate position between the Y-coordinate of the left foot positionP2 and the Y-coordinate of the right foot position P3 may be calculated.

The game processing device 50 calculates a corrected distance Ly2 bymultiplying each distance Ly by a predetermined ratio R2. The ratio R2is larger than 0 and smaller than 1. Next, the game processing device 50sets a rear boundary 102 at the position that is located away rearward,which is in the direction opposite to the X-direction, by the correcteddistance Ly2 from the left foot position P2 or the right foot positionP3, which serves as a reference position. When the distance Ly betweenthe seating position P1 and the left foot position P2 and the distanceLy between the seating position P1 and the right foot position P3 areboth calculated, these distances Ly may be used to set a rear boundary102 for the left foot and a rear boundary 102 for the right foot.

The rear boundary 102 simply needs to indicate the boundary of theX-coordinates. For example, the rear boundary 102 may be a Y-Z planethat is parallel to the Y-direction and the Z-direction with theX-direction set as a normal direction. In one example, the ratio R2 isdetermined taking into account, for example, a general angle obtainedwhen the user 200 seated on the seat 15 bends the leg rearward (in thedirection opposite to the X-direction) by rotating the lower leg withrespect to the knee as a preparatory motion for an action of kickingaway the shoe object in the virtual space. Basically, the ratio R2 isfixed. Thus, for example, the rear boundary 102 is set at a positionlocated farther from the reference position for a user 200 having a longlower leg than a user 200 having a short lower leg.

In addition, the game processing device 50 calculates a correcteddistance Ly1 by multiplying each distance Ly by a predetermined ratioR1. The ratio R1 is larger than 0 and smaller than 1. The ratio R1 maybe different from the ratio R2 or may be the same as the ratio R2. Whilethe ratios R1 and R2 are basically fixed, the ratios R1 and R2 can befitted to users having diverse body forms by setting the ratios R1 andR2 to be adjustable. Subsequently, the game processing device 50 sets afront boundary 101 at the position that is located away frontward, whichis in the X-direction, by the corrected distance Ly1 from the left footposition P2 or the right foot position P3, which serves as the referenceposition. When calculating both the distance Ly in the Y-directionbetween the seating position P1 and the left foot position P2 and thedistance Ly in the Y-direction between the seating position P1 and theright foot position P3, the game processing device 50 may use thesedistances to set a front boundary 101 for the left foot and a frontboundary 101 for the right foot.

The front boundary 101 simply needs to indicate the boundary of theX-coordinates. For example, the front boundary 101 may be a Y-Z planethat is parallel to the Y-direction and the Z-direction with theX-direction set as the normal direction. In one example, the ratio R1 isdetermined taking into account, for example, a general angle obtainedwhen the user 200 seated on the seat 15 kicks frontward (in theX-direction) by rotating the lower leg with respect to the knee as amotion for the action of kicking away the shoe object in the virtualspace. Thus, for example, the front boundary 101 is set at a positionlocated farther from the reference position for a user 200 having a longlower leg than a user 200 having a short lower leg.

The front boundary 101 and the rear boundary 102 set in such a mannerreflect the length of the lower leg, which is the body form of the user.For example, in a case in which it is determined whether to execute anaction based on changes in the movement direction of the ankle positionof the user, the action may be executed at a timing against the user'sintention such as a timing at which the user moves the feet a little. Insuch a case, the user may experience discomfort. Further, when theboundary or the like for determining whether to execute an action isuniform among users, the action is executed easily for some users andthe action is not executed easily for other users. In the presentembodiment, the boundary is set in accordance with the body form of auser. This allows the action to be executed following the user'sintention and reduces the discomfort of the user.

The procedure of using the front boundary 101 and the rear boundary 102for the game to progress will now be described with reference to FIGS. 4and 5. As described above, back-and-forth swinging of the seat 15 islimited such that the seat 15 swings slightly in accordance with themotion of the user 200. FIGS. 4 and 5 show only the right foot.

As shown in FIG. 4, when starting the kick-the-shoe-on-the-swing game,as preparation for kicking frontward, the user 200 rotates the lower legwith respect to the knee such that the lower leg is located below theseat 15. The game processing device 50 determines the left foot positionP2 (not shown) and the right foot position P3 to determine whether atleast one of the left foot position P2 and the right foot position P3has crossed the rear boundary 102.

When determining that at least one of the left foot position P2 and theright foot position P3 has crossed the rear boundary 102, the gameprocessing device 50 determines that the kick-the-shoe-on-the-swingmotion has been started.

After determining that the kick-the-shoe-on-the-swing motion has beenstarted, the game processing device 50 determines at least one of theleft foot position P2 and the right foot position P3 has crossed thefront boundary 101 while determining the left foot position P2 and theright foot position P3.

As shown in FIG. 5, when determining that at least one of the left footposition P2 (not shown) and the right foot position P3 has crossed thefront boundary 101, the game processing device 50 determines that thekick-the-shoe-on-the-swing motion is complete. Then, in the virtualspace, the game processing device 50 causes the shoe object of thevirtual character to move toward the front of the user 200, which is inthe kick-out direction, and then fall in accordance with a predeterminedtrajectory. More specifically, the game processing device 50 causes theshoe object to move and then fall in accordance with the kick-outdirection of the user based on the detection data of the tracking sensor22. For example, when the kick-out direction is oriented toward thefront left as viewed from the user 200, the game processing device 50draws the shoe object to be moved and fallen toward the front left.

The game processing device 50 obtains the speed of the tracking sensor22 when the kick-the-shoe-on-the-swing motion is complete. This speed isat least one of the speed, angular velocity, acceleration, and the like,which refer to the movement distance per unit of time. The gameprocessing device 50 causes the shoe object to move based on data inwhich the speed is associated with the movement distance (flyingdistance) of the object. The speed may be calculated from the distancebetween a pair of the tracking sensors 22. Alternatively, the speed maybe detected using a speed detection sensor incorporated in the trackingsensor 22 or using an external speed detection sensor. When the speed iscalculated from the distance, a distance Ly between the tracking sensor22 attached to the seat 15 and the left foot position P2 or the rightfoot position P3 is divided by a time T, during which the lower leg isrotated, to obtain the speed V (V=Ly/T). The time T may be a preset time(constant). Alternatively, the time from when the foot crosses the rearboundary 102 to when the foot crosses the front boundary 101 may bemeasured, and the measured time may be used as the time T.

FIG. 6A is a table serving as an example of data in which the movementspeed of the foot is associated with the movement distance of the shoeobject. In this table, the speed is associated with the distance suchthat the distance increases in stages as the speed increases. Further,as shown in the map of FIG. 6B, the speed may be associated with thedistance such that the distance increases in a continuous manner as thespeed increases. Instead of or in addition to the table, the gameprocessing device 50 may calculate the movement distance using apredetermined arithmetic expression.

FIG. 7 schematically shows an exemplary view 110, which is visuallyrecognized by the user 200. The view 110 displays the image of a virtualspace 111. The virtual space 111 displays a shoe object 112 and part ofa virtual character 113.

After a single kick-the-shoe-on-the-swing motion is complete and theshoe object 112 moves, the game processing device 50 displays the shoeobject 112 again on the foot of the virtual character, thereby enablingthe user to perform the kick-the-shoe-on-the-swing motion again. Whenthe fall position or movement trajectory of the shoe object 112satisfies a score assigning condition, a score is assigned to the user.Further, a score to be assigned may differ depending on the fallposition or movement trajectory. Instead, when the fall position ormovement trajectory of the shoe object 112 satisfies an achievementcondition, the mission is complete.

In this manner, the user is actually seated on the seat 15 of the swingto perform the kick-the-shoe-on-the-swing motion. Thus, the motion ofthe user causes the seat 15 to swing. Further, when the swinging isdetected by the HMD 30, the game processing device 50 moves the imagedisplayed in the HMD 30 in accordance with the actual swinging. Thisreduces VR motion sickness and allows the user to have a sense ofimmersion as if the user performs the kick-the-shoe-on-the-swing motionriding on a virtual swing.

The procedure of the game processing of the present embodiment will nowbe described.

The procedure of setting a boundary will now be described with referenceto FIG. 8. The game processing device 50 obtains detection data from thetracking sensor 22 and the like to determine the position of thetracking sensor 22 (sensor position) (step S1). More specifically, thegame processing device 50 determines the seating position P1, the leftfoot position P2, and the right foot position P3.

When determining that a new game is being started or that a new user isstarting the game, the game processing device 50 sets a boundary basedon the position of the tracking sensor 22 (step S2). More specifically,the seating position P1 and at least one of the left foot position P2and the right foot position P3 are used to set the front boundary 101and the rear boundary 102. The front boundary 101 and the rear boundary102 may be common to both feet or may be set separately for each foot.The position information of the front boundary 101 and the positioninformation of the rear boundary 102 are stored as the settinginformation 61 in the data memory 60. The position information of thefront boundary 101 and the position information of the rear boundary 102may be stored in association with identification information of theuser. When the game is played by the user who has set the positioninformation of the front boundary 101 and the position information ofthe rear boundary 102, the position information of the front boundary101 and the position information of the rear boundary 102 may be readfrom the data memory 60. The game may be played using the read positioninformation.

The procedure of causing the game to progress will now be described withreference to FIG. 9. The game processing device 50 starts the game basedon a trigger such as an input operation by the input operation device 65(step S11). For example, the game processing device 50 performsinitialization such as resetting of the position information of theboundary used in the last game. Further, the game processing device 50obtains the detection data of the measurement device 32 of the HMD 30,and displays the image of a display range corresponding to the detectiondata.

While causing the game to progress based on the game progress data 63,the game processing device 50 determines whether the left foot positionP2 or the right foot position P3 has crossed the rear boundary 102 basedon the position information of the rear boundary 102 (step S12). Whendetermining that the left foot position P2 or the right foot position P3has not crossed the rear boundary 102 (step S12: NO), the gameprocessing device 50 proceeds to step S12.

When determining that the left foot position P2 or the right footposition P3 has crossed the rear boundary 102 (step S12: YES), the gameprocessing device 50 determines that the corresponding leg has startedthe kick-the-shoe-on-the-swing motion (step S13).

While causing the game to progress, the game processing device 50determines whether the left foot position P2 or the right foot positionP3 has crossed the front boundary 101 (step S14). When determining thatthe left foot position P2 or the right foot position P3 has not crossedthe front boundary 101 (step S14: NO), the game processing device 50proceeds to step S17. In step S14, it may be determined whether the leftfoot position P2 or the right foot position P3 has crossed the frontboundary 101 within a predetermined time such as one or two seconds.When determining that the left foot position P2 or the right footposition P3 has not crossed the front boundary 101 within thepredetermined time (step S14: NO), the game processing device 50 mayproceed to step S17.

When determining that at least one of the left foot position P2 and theright foot position P3 has crossed the front boundary 101 (step S14:YES), the game processing device 50 determines that thekick-the-shoe-on-the-swing motion is complete (step S15).

Subsequently, the game processing device 50 executes the action ofkick-the-shoe-on-the-swing motion in the virtual space (step S16). Inthis case, the game processing device 50 uses the map of FIG. 6B tocalculate the movement distance of the shoe object from the movementspeed. For example, the game processing device 50 draws the leg of theleg of the virtual character and the shoe object, and draws the shoeobject to be moved and then fallen along a parabola in the kick-outdirection in accordance with the calculated movement distance. Further,the game processing device 50 draws a new shoe object on the foot of thevirtual character after the shoe object moves away from the foot of thevirtual character.

The game processing device 50 determines whether the game has endedbased on the game end condition (step S17). When determining that theend condition is satisfied (step S17: YES), the game processing device50 ends the game. When the end condition is not satisfied (step S17:NO), the game processing device 50 returns to the step S12.

The first embodiment has the following advantages.

(1) The distance between the tracking sensor 22 attached to the foot ofthe user and the tracking sensor 22 attached to the housing 11 iscalculated to set the front boundary 101 and the rear boundary 102 basedon the calculated distance. Thus, the positions of the front boundary101 and the rear boundary 102 are set in accordance with the body formof the user. Further, when the positional relationship between thetracking sensor 22 attached to the user and the front boundary 101 orthe rear boundary 102 satisfies the predetermined condition, the actioncorresponding to the motion of the user is executed in the virtualspace. The position of the real world where the motion of the user isstarted differs depending on the body form of the user. Thus, the use ofthe boundary reflecting the body form of the user restricts the actionfrom being executed against the user's intention, thereby reducing thediscomfort of the user.

(2) The game processing device 50 obtains the speed when the trackingsensor 22 attached to the foot of the user crosses the front boundary101. Further, the magnitude of the speed is reflected on the movementdistance of the shoe object of the kick-the-shoe-on-the-swing motion.This allows the user to adjust the movement distance of the shoe objectin the virtual space by adjusting the movement speed of the foot. Thus,the game can be diversified, for example, scores are assigned inaccordance with the fall position of the shoe object and scores areassigned in accordance with the movement distance.

(3) The game processing device 50 moves the shoe object from the foot ofthe virtual character in the virtual space when the tracking sensor 22attached to the foot of the user crosses the front boundary 101. Thus,the timing at which the movement of the object is started can beapproximated to a timing that is intended by the user. Thus, the usercan intuitively perform the motion of kicking as an input operation.

(4) The tracking sensor 22 is attached to the foot of the user. Thisallows the motion of the leg to be reflected on an action in the virtualspace. Thus, a novel game can be provided.

Second Embodiment

A system of displaying an image according to a second embodiment willnow be described. The second embodiment is the same as the firstembodiment in the method for setting a boundary. The second embodimentis different from the first embodiment in how the set boundary is used.Like or the same reference numerals are given to those components thatare like or the same as the corresponding components of the firstembodiment.

In the second embodiment, the front boundary 101 and the rear boundary102 are used to determine whether to start moving an object in thevirtual space. More specifically, when the positional relationshipbetween the left foot position P2 and the right foot position P3 with atleast one of the front boundary 101 and the rear boundary 102 satisfiesa predetermined condition, the game processing device 50 moves a seatobject, which corresponds to the seat 15, and the virtual character.Further, a user perspective in the virtual space corresponds to theperspective of the virtual character. Thus, as the virtual charactermoves, the user perspective in the virtual space moves. Conventionalmethods for moving a virtual character include a method for causing thevirtual character and the like to move in the virtual space as the userperspective moves in the real world and a method for causing the virtualcharacter and the like to automatically move in accordance with a gamestory. Alternatively, for example, the user in the real world operates acontroller, which is held by himself or herself, and uses a graphicaluser interface (GUI) such as an icon in the virtual space to instructthe virtual character to move. In the former case, the virtual charactermay move against the user's intention. In the latter case, movement ofthe virtual character toward a target position requires a movementinstruction using a GUI such as in icon that does not exist in the realworld by operating the controller. This may reduce a sense of immersion.In the present embodiment, the user causes the virtual character to moveby moving the legs while riding on the swing. This allows the user tomove the virtual character with motions that are more intuitive than theconventional methods. Instead of or in addition to the seat object andthe virtual character, the object to be moved may be the entirety of ahousing object in the virtual space, which corresponds to the housing11. Further, in the present embodiment, movement of the seat object andthe virtual character is determined in a mode (fly mode), which differsfrom a mode of performing the kick-the-shoe-on-the-swing motion (kickmode).

The front boundary 101 and the rear boundary 102 shown in FIGS. 10A to12 simply need to indicate the boundary in the X-direction. For example,the front boundary 101 and the rear boundary 102 are Y-Z planes in whichthe X-direction is a normal direction. The game processing device 50determines whether the left foot position P2 (not shown) and the rightfoot position P3 (not shown) have both crossed the same boundary.

As shown in FIG. 10A, when determining that both feet have crossed thefront boundary 101, the game processing device 50 determines thedirection in which the user 200 has stretched the legs. When determiningthat the leg-stretching direction is the X-direction, which is orientedto the front surface of the user, the game processing device 50 causesthe seat object and the virtual character in the virtual space to moverearward (in the direction opposite to X-direction), which is thedirection opposite to the leg-stretching direction. In this case, thegame processing device 50 may cause the seat object and the virtualcharacter 113 to horizontally move rearward or to swing rearward. Inthis manner, the object is moved and displayed in the direction oppositeto the direction in which the user has thrusted the legs. The reactionof the motion of the virtual character produces thrust, thereby causingthe user to imagine as if the seat object and the virtual character aremoving.

The method by which the game processing device 50 determines thedirection in which the user has stretched the legs is not particularlylimited. For example, the leg-stretching direction may be determinedwith reference to the left foot position P2 when setting the boundarybefore the game starts. Instead, the direction in which the left footposition P2 has been stretched or the direction in which the right footposition P3 has been stretched may be determined with reference to theseating position P1. Alternatively, the leg-stretching direction may bedetermined by calculating the median value of the left foot position P2and the right foot position P3 and using the direction from the seatingposition P1, which serves as a reference, to the median value.

As shown in FIG. 10B, when determining that the left foot position P2(not shown) and the right foot position P3 (not shown) have both crossedthe front boundary 101 and the user 200 has stretched both legs towardthe front left, the game processing device 50 causes the seat object andthe virtual character 113 to move toward the rear right in the virtualspace.

As shown in FIG. 10C, when determining that the left foot position P2(not shown) and the right foot position P3 (not shown) have both crossedthe front boundary 101 and the user 200 has stretched both legs towardthe front right, the game processing device 50 causes the seat objectand the virtual character 113 to move toward the rear left in thevirtual space.

In FIGS. 10A and 10B, the seat object and the virtual character 113 movein the direction opposite to the direction in which the user hasstretched the legs. Instead, the seat object and the virtual character113 may move in the same direction as the leg-stretching direction. Forexample, when the user thrusts both legs toward the front of him or her,the seat object and the virtual character 113 may move frontward.Further, when the user has stretched both legs toward the front left,the seat object and the virtual character 113 may move toward the frontleft.

Further, the seat object and the virtual character 113 do not have tomove in the above-described direction. For example, the seat object andthe virtual character 113 may move in the right direction or in the leftdirection (immediately beside the user). For example, when the userstretches the legs by an angle that is greater than or equal to apredetermined angle from the front, which serves as a reference, towardthe right, the seat object and the virtual character 113 may move in theright direction. Further, when the user stretches the legs by an anglethat is greater than or equal to the predetermined angle toward theleft, the seat object and the virtual character 113 may move in the leftdirection.

As shown in FIG. 11A, when determining that the user 200 has bent thelegs to position the lower legs below the seat 15 and the left footposition P2 (not shown) and the right foot position P3 (not shown) havecrossed the rear boundary 102, the game processing device 50 determinesthe direction of the lower legs. When determining that the direction inwhich the lower legs stretch is opposite to the X-direction, the gameprocessing device 50 causes the seat object and the virtual character113 to move frontward, which is in the X-direction, in the virtualspace.

As shown in FIG. 11B, when determining that the left foot position P2(not shown) and the right foot position P3 (not shown) have both crossedthe rear boundary 102, the game processing device 50 determines thedirection of the lower legs. When determining that the lower legs aredirected toward the rear right, the game processing device 50 causes theseat object and the virtual character to move toward the front left.

As shown in FIG. 11C, when determining that the left foot position P2(not shown) and the right foot position P3 (not shown) have both crossedthe rear boundary 102, the game processing device 50 determines thedirection of the lower legs. When determining that the lower legs aredirected toward the rear left, the game processing device 50 causes theseat object and the virtual character to move toward the front right.

When the user 200 bends the right leg while stretching the left leg andbends the left leg while stretching the right leg to kick out the leftleg and the right leg alternately, the seat object and the virtualcharacter move in the Y-direction, which is oriented toward the upperside, in the virtual space.

As shown in FIG. 12, when determining that the left foot position P2(not shown) has crossed the front boundary 101 and the right footposition P3 (not shown) has crossed the rear boundary 102, the gameprocessing device 50 draws the seat object and the virtual characterthat move upward. While a state in which one of the feet is over thefront boundary 101 and the other one of the feet is over the rearboundary 102 is repeated within a predetermined time, drawing of theseat object and the virtual character that move upward is continued.When the number of times one of the feet has crossed the front boundary101 and the other one of the feet has crossed the rear boundary 102 hasreached a predetermined number of times, the seat object and the virtualcharacter 113 that move upward may start to be drawn.

The procedure of the game processing according to the second embodimentwill now be described with reference to FIG. 13.

When starting the game (step S20), the game processing device 50performs initialization such as resetting of the history of the lasttime. Then, the game processing device 50 determines whether both feethave crossed the same boundary (step S21).

When determining that both feet have crossed one of the front boundary101 and the rear boundary 102 (step S21: YES), the game processingdevice 50 obtains the direction in which the legs (lower legs) have beenstretched (step S22). Subsequently, the game processing device 50displays, on the HMD 30, an image in which the seat object and thevirtual character have been moved based on the obtained direction (stepS23).

In step S21, when determining that both feet of the user have notcrossed the same boundary (step S21: NO), the game processing device 50determines whether each of the feet has crossed a different boundary(step S25). When determining that each of the feet of the user hascrossed a different boundary (step S25: YES), the game processing device50 displays, on the HMD 30, an image in which the seat object and thevirtual character have been moved upward (step S26), thereby proceedingto step S24. When determining that each of the feet has not crossed adifferent boundary (step S25: NO), the game processing device 50proceeds to step S24.

The game processing device 50 determines whether the game has been ended(step S24). When determining that the game has been ended (step S24:YES), the process is ended. When determining that the game has not beenended (step S24: NO), the process returns to step S21.

The second embodiment has the following advantages.

(5) When the user stretches or bends the legs such that thepredetermined condition is satisfied by the positional relationship ofthe left foot position P2 and the right foot position P3 with at leastone of the front boundary 101 and the rear boundary 102, the seat objectand the virtual character move. This allows the user to move the virtualcharacter in the virtual space with intuitive operations at his/her ownintentional timing.

(6) When the left foot position P2 and the right foot position P3crosses the front boundary 101, the game processing device 50 causes theseat object and the virtual character to move rearward, that is, in thedirection opposite to the direction in which both legs are stretched.Further, when the left foot position P2 and the right foot position P3cross the rear boundary 102, the game processing device 50 causes theseat object and the virtual character to move forward, that is, in thedirection opposite to the direction in which both legs are bent. Suchuse of the boundary reflecting the body form of the user restricts theaction from being executed against the user's intention.

(7) The game processing device 50 changes how the seat object and thevirtual character move depending on the difference in the positionalrelationship with the boundary of the left foot position P2 and theright foot position P3. That is, when the left foot position P2 and theright foot position P3 both cross the same boundary, the game processingdevice 50 causes the seat object and the virtual character to moverearward or frontward. By contrast, when the left foot position P2 andthe right foot position P3 cross different boundaries, the gameprocessing device 50 causes the seat object and the virtual character tomove upward. Thus, the boundary can be used for determining the types ofactions.

The above-described embodiments may be modified as follows. Theabove-described embodiments and the following modifications can becombined as long as the combined modifications remain technicallyconsistent with each other.

In each of the above-described embodiments, the tracking sensor 22 isattached around the ankle of the user. Instead of or in addition tothis, the value of detection data of the tracking sensor 22 attached toa position of the body of the user other than around the ankle may bereflected on an action. For example, in a flying mode, the movementspeeds, movement distances, and the like of the seat object and thevirtual character may be increased based on detection data of thetracking sensor 22 attached to the position of any part of the upperbody including the head of the user. Further, in the kick mode, themotion of the upper body may be reflected on the action. For example,the movement distance of the shoe object may be increased in accordancewith the motion of the upper body of the user. Furthermore, in theflying mode, in a case in which the left foot position P2 and the rightfoot position P3 cross the front boundary 101 and the position of thetracking sensor 22 attached to the head of the user or the position ofthe tracking sensor 22 attached to a position of the upper body otherthan the head moves to a position located rearward from an initialposition when starting the game, that is, in a case in which the head orthe upper body other than the head is inclined in the direction oppositeto the leg-stretching direction, the action may be reflected byincreasing the movement speeds, movement distances, and the like of theseat object and the virtual character.

In each of the above-described embodiments, the tracking sensor 22attached around the ankle of the user is used to determine whether tostart executing an action and whether to complete the executed action.Instead of or in addition to this, the tracking sensor 22 attached to aposition of the body of the user other than around the ankle may be usedto determine at least one of whether to start executing an action andwhether to complete the executed action.

An exemplary procedure of executing an action of throwing an object inthe virtual space using the tracking sensor 22 attached to the arm willnow be described with reference to FIGS. 14A to 14C. As shown in FIG.14A, the tracking sensors 22 are attached to the elbow and wrist of theuser 200, respectively. The game processing device 50 determines a wristposition P5 and an elbow position P6 based on the data obtained from atleast one of the tracking sensors 22 and the tracking assistance devices21. Based on the distance between the tracking sensors 22, the gameprocessing device 50 calculates the length from the elbow to the wristof the user 200, that is, a length Lx1 of the forearm. Further, the gameprocessing device 50 sets the rear boundary 102 at a position locatedtoward the shoulder of the user 200 and away from the wrist position P5by a length obtained by multiplying the length Lx1 by a predeterminedratio R5. The ratio in this example is greater than or equal to 1. Inaddition, the game processing device 50 sets the front boundary 101 at aposition toward the elbow and away from the wrist position P5 by alength obtained by multiplying the length Lx1 by a predetermined ratioR6.

As shown in FIG. 14B, the game processing device 50 determines whetherthe wrist position P5 has crossed the rear boundary 102. Whendetermining that the wrist position P5 has crossed the rear boundary102, the game processing device 50 determines that the motion hasstarted. The action corresponding to this motion is, for example, tothrow an object like a ball or cast an object such as a fishing lure ina water surface. Instead of throwing an object, for example, the actionmay be bending the arm such as hitting a ball or dancing.

As shown in FIG. 14C, the game processing device 50 determines whetherthe wrist position P5 has crossed the front boundary 101 after themotion starts. When determining that the wrist position P5 has crossedthe front boundary 101, the game processing device 50 determines thatthe motion is complete. Next, the game processing device 50 causes theaction to be reflected in the virtual space. For example, in the case ofcasting, the game processing device 50 causes a lure to move frontwardin accordance with the calculated trajectory. In the case of throwing aball, the game processing device 50 causes the ball to move frontward inaccordance with the calculated trajectory.

Instead of the length of the forearm, the length of the entire arm orthe length of the upper arm may be used to set a boundary. To calculatethe length of the entire arm, the tracking sensors 22 are attached tothe wrist and the shoulder. To calculate the length of the upper arm,the tracking sensors 22 are attached to the elbow and the shoulder. Thegame processing device 50 calculates the length of the entire arm or thelength of the upper arm and multiplies the calculated length by apredetermined ratio to set a boundary with reference to the position ofthe wrist, shoulder, or elbow. The boundary set in this manner can beused to determine whether to execute the motion of rotating the entirearm or an action corresponding to the motion of bending the arm. Thedetermination of whether movement has started, determination of whethermovement is complete, determination of the type of motion, and the likemay be performed in a game of, for example, playing golf, baseball,tennis, and table tennis.

In each of the above-described embodiments, the game provided by thesystem is played by a user in a sitting position. Instead, the user mayplay the game in a standing position.

As shown in FIG. 15A, the game processing device 50 calculates a lengthLy5 of the lower leg based on the position of the tracking sensor 22.Further, the game processing device 50 sets a boundary 120 at a positionlocated toward the knee of the user 200 and away from an ankle positionP7 by a length obtained by multiplying the length Ly5 of the lower legby a predetermined ratio R7. The ratio R7 in this example is less than1.

As shown in FIG. 15B, in a game including an action of kicking an objectsuch as soccer, the game processing device 50 may determine that theaction intended by the user is dribbling when the user starts the motionof kicking from the ankle position P7, which is lower than the boundary120.

As shown in FIG. 15C, the game processing device 50 may determine thatthe action intended by the user is shooting when the user starts themotion of kicking from the ankle position P7, which is higher than theboundary 120.

In the second embodiment, a predetermined action is executed in thevirtual space when a predetermined condition is satisfied by thepositional relationship between the tracking sensor 22 attached aroundthe ankle and the boundary that has been set in advance. Instead of orin addition to this, the virtual character or the like may be movedusing the tracking sensor 22 attached to a position of the body of theuser other than around the ankle. For example, the virtual character orthe like may be moved when the predetermined condition is satisfied bythe positional relationship between the tracking sensor 22 attached tothe position of any part of the upper body and the boundary set for theupper body. Alternatively, the virtual character or the like may bemoved when the predetermined condition is satisfied by the positionalrelationship between the tracking sensor 22 attached to the head and theboundary set for the head. The movement of the virtual character or thelike may be any one of, for example, upward movement, downward movement,leftward movement, and rightward movement. In another option, themovement of the virtual character or the like may be movement in themovement direction of the upper body or movement in other directions.

In each of the above-described embodiments, it is determined that amotion has started when the left foot position P2 or the right footposition P3 crosses the rear boundary 102, and it is determined that themotion is complete when the left foot position P2 or the right footposition P3 crosses the front boundary 101. Instead, when the left footposition P2 or the right foot position P3 crosses the rear boundary 102,the kick-the-shoe-on-the-swing motion or the like may be executed. Asanother option, when the left foot position P2 or the right footposition P3 crosses the front boundary 101, thekick-the-shoe-on-the-swing motion or the like may be executed. Thisreduces processing load in the determination process for an action.

In the second embodiment, while a state in which one of the feet is overthe front boundary 101 and the other one of the feet is over the rearboundary 102 is repeated within a predetermined time, the seat objectand the virtual character that move upward are drawn. Instead of or inaddition to this, when the positional relationship between the trackingsensor 22 of one of the feet and the front boundary 101 satisfies apredetermined condition and the positional relationship between thetracking sensor 22 of the other one of the feet and the rear boundary102 satisfies a predetermined condition, a seat object and a virtualcharacter that are moving downward may be drawn. Alternatively, when thepositional relationship between the tracking sensor 22 of one of thefeet and the front boundary 101 or the rear boundary 102 satisfies apredetermined condition or when the positional relationship between thetracking sensors 22 of both feet and the front boundary 101 or the rearboundary 102 satisfies a predetermined condition, a seat object and avirtual character that are moving downward may be drawn. The motion ofthe user is not particularly limited. Instead of moving the legsalternately, for example, the motion may be to move both legssimultaneously back and forth, move both legs simultaneously from sideto side, or pedal a bike using both legs. Alternatively, instead of oneof upward movement and downward movement, a seat object and a virtualcharacter that repeatedly move upward and downward may be drawn.

In each of the above-described embodiments, the movement distance of theshoe object may be changed in accordance with the length of the lowerleg. For example, the movement distance of the shoe object may becorrected to be longer when the lower leg is long than when the lowerleg is short. When the speed is fixed, the centrifugal force produced byrotating the lower leg with respect to the knee increases as the lengthof the lower leg increases. Thus, when the movement distance of the shoeobject is increased as the length of the lower leg increases, the motionbecomes close to a phenomenon in the real world. Conversely, themovement distance of the shoe object may be corrected to be shorter whenthe lower leg is long than when the lower leg is short.

In each of the above-described embodiments, when at least one of theleft foot position P2 and the right foot position P3 crosses the rearboundary 102, it is determined that the motion corresponding to thekick-the-shoe-on-the-swing motion has started. Instead, when at leastone of the left foot position P2 and the right foot position P3 is onthe rear boundary 102, it may be determined that the motion has started.Likewise, when at least one of the left foot position P2 and the rightfoot position P3 crosses the front boundary 101, it is determined thatthe motion corresponding to the kick-the-shoe-on-the-swing is complete.Instead, when at least one of the left foot position P2 and the rightfoot position P3 is on the front boundary 101, it may be determined thatthe motion is complete.

In each of the above-described embodiments, the seat 15, which is aswing, slightly swings in the front-rear direction. Instead, the seat 15may be swung back and forth when the user performs the motion of ridingon the swing. This allows for determination of thekick-the-shoe-on-the-swing motion of the user by moving the frontboundary 101 and the rear boundary 102 with reference to the seatingposition P1.

In each of the above-described embodiments, the length from the seatingposition P1 to the left foot position P2, which is the ankle position,or the length from the seating position P1 to the right foot positionP3, which is the ankle position, is calculated as the length of thelower leg. Instead, when the height of the seat 15 is changed byadjusting the hanging portion 13 such that the foot of the user 200 isin contact with the floor surface 500, the height of the seatingposition P1 may be calculated as the length of the lower leg.

In each of the above-described embodiments, the length of the lower legof the user is calculated. Instead, the length of the thigh of the usermay be calculated. The length of the thigh can be approximated betweenthe coordinates of the seating position P1 and the coordinates of theleft foot position P2 or the right foot position P3. When the thigh islong, the length of the lower leg is expected to be long. Thus, theboundary may be set based on the length of the thigh. In this case, theboundary can be set in accordance with the body form of the user.

In each of the above-described embodiments, the boundary is set withreference to the left foot position P2 or the right foot position P3.Instead of or in addition to this, the boundary may be set withreference to a tracking sensor 22 that is not attached to the body ofthe user, such as the tracking sensor 22 attached to the seat 15.

In each of the above-described embodiments, it is determined that themotion has started when the foot position crosses the rear boundary 102and it is determined that the motion is complete when the foot positioncrosses the front boundary 101. Instead of or in addition to this, thetype of motion of the user may be determined based on the positionalrelationship between the leg or arm and the boundary. For example, in abaseball game, the motion of hitting, the motion of pitching, and thelike may be determined based on the positional relationship between thearm and the boundary. In a dance game, turning, jumping, and the likemay be determined based on the positional relationship between the legor arm and the boundary.

In each of the above-described embodiments, the front boundary 101 andthe rear boundary 102 are set with reference to the foot of the user.Instead of or in addition to this, an upper boundary and a lowerboundary that indicate the boundary in the Y-direction may be set inFIG. 3. Alternatively, a left boundary and a right boundary thatindicate the boundary in the Z-direction may be set. Further, the numberof boundaries may be one or may be three or more. In addition, theboundary does not have to be a plane but may be a dot or a line.

In each of the above-described embodiments, when the left foot positionP2 or the right foot position P3 of the user crosses any one of thefront boundary 101 and the rear boundary 102, it is determined that thepositional relationship satisfies a predetermined condition. Instead,the game processing device 50 may determine whether to execute an actionbased on whether the foot position is moving toward or away from theboundary.

In each of the above-described embodiments, the tracking system 20,which serves as one example, causes the tracking assistance devices 21,which serve as laser oscillators, and the tracking sensors 22, whichserve as optical receivers, to cooperate with each other. As long as thepositions and orientations of the tracking sensors 22 are detectable,other tracking systems 20 may be employed. For example, an outside-intracking system other than the above-described embodiments may be used.In the outside-in tracking system, for example, an assistance device,which serves as an optical receiver, is provided in a gameplay space.The assistance device cooperates with the tracking sensor 22, whichserves as a laser oscillator, to calculate the position and orientationof the tracking sensor 22. Instead, an inside-out tracking system may beused. This system is a sensor incorporated in the tracking sensor 22 todetermine the position of the user by scanning the real space.

In each of the above-described embodiments, the HMD 30, which includesan optical receiver, and the tracking assistance devices 21, whichinclude laser oscillators, cooperate with each other to detect theposition and orientation of the HMD 30. Instead, a HMD 30 including alaser oscillator and tracking assistance devices 21 including opticalreceivers may cooperate with each other to detect the position andorientation of the HMD 30. Alternatively, the inside-out tracking systemmay be used.

In each of the above-described embodiments, the display used by a user,which serves as an example, is the HMD 30. Instead of or in addition tothis, the display does not have to be attached to the body of the user.The display may be, for example, an installment-type display, a displayfor a mobile game device, or a display incorporated in an arcade game.When such displays are used, signals based on the operation of the usermay be received from the input operation device (controller) to changethe display range in the display.

In each of the above-described embodiments, the HMD 30 is a deviceseparate from the game processing device 50. Instead, the gameprocessing device 50 may be equipped with the HMD 30. Further, the HMD30 may be a standalone HMD including a device that detects its position.

In each of the above-described embodiments, one example of the housingis a swing. Instead, the housing may include a seat on which the user isseated. Alternatively, in the housing, the user may perform a motion ina standing or lying position. The housing may be, for example, a vehiclesuch as an automobile, bicycle, airplane or flying object, submarine,roller coaster, or rocket. Instead, the housing may have a gun orbazooka that can shoot. As another option, the housing may be playequipment such as a slide. Alternatively, the system may provide a gamein which the housing 11 is not used or provide an application other thana game.

In each of the above-described embodiments, the system is a system withwhich the user can play a game. Instead, the system may be, for example,a system for giving experience to the user, a system used to viewcontents such as a movie, a system that allows the user to interact withother users, a learning system, a training system, or a simulationsystem for the medical field.

Various changes in form and details may be made to the examples abovewithout departing from the spirit and scope of the claims and theirequivalents. The examples are for the sake of description only, and notfor purposes of limitation. Descriptions of features in each example areto be considered as being applicable to similar features or aspects inother examples. Suitable results may be achieved if sequences areperformed in a different order, and/or if components in a describedsystem, architecture, device, or circuit are combined differently,and/or replaced or supplemented by other components or theirequivalents. The scope of the disclosure is not defined by the detaileddescription, but by the claims and their equivalents. All variationswithin the scope of the claims and their equivalents are included in thedisclosure.

What is claimed is:
 1. A system for displaying an image of a virtualspace on a display, the system comprising circuitry configured to:obtain detection data from a plurality of position detection sensorscomprising a first position detection sensor and a second positiondetection sensor; detect a first position by the first positiondetection sensor; detect a second position by the second positiondetection sensor; calculate a distance between the first position andthe second position; set, based on the distance, a boundary fordetermining a motion of a user, the boundary dividing a first regionfrom a second region; determine, when the motion of the user occurs,whether the first position moves from the first region to the secondregion over the boundary or whether the first position moves from thefirst region to the boundary; and responsive to the first positionmoving from the first region to the second region over the boundary orthe first position moving from the first region to the boundary, move anobject in the virtual space.
 2. The system according to claim 1, whereinthe first position detection sensor is attached to the user.
 3. Thesystem according to claim 1, wherein the motion of the user includes amotion in a first direction, the boundary comprises a first boundarythat is away from the first position detection sensor in the firstdirection, and the circuitry is configured to, responsive to the firstposition moving from the first region to the second region over thefirst boundary or the first position moving from the first region to thefirst boundary, determine that the motion of the user has started. 4.The system according to claim 1, wherein the motion of the user includesa motion in a second direction, the boundary comprises a second boundarythat is away from the first position detection sensor in the seconddirection, and the circuitry is configured to, responsive to the firstposition moving from the first region to the second region over thesecond boundary or the first position moving from the first region tothe second boundary, determine that the motion of the user is complete.5. The system according to claim 4, wherein the circuitry is configuredto obtain, based on the detection data from the position detectionsensors or detection data from a speed detection sensor attached to theuser, a speed of the first position detection sensor when the firstposition crosses or reaches the second boundary and to reflect theobtained speed on the movement of the object.
 6. The system according toclaim 1, wherein the boundary is away from the first position detectionsensor in a movement direction of the first position, and the circuitryis configured to, responsive to the first position moving from the firstregion to the second region over the boundary or the first positionmoving from the first region to the boundary, move the object in adirection corresponding to the movement direction of the first positionin the virtual space.
 7. The system according to claim 1, wherein themovement of the object differs depending on the positional relationshipbetween the first position and the boundary.
 8. The system according toclaim 1, wherein at least the first position detection sensor isattached to a foot of the user.
 9. The system according to claim 1,wherein the circuitry is configured to, responsive to the first positionmoving from the first region to the second region over the boundary orthe first position moving from the first region to the boundary, movethe object in a front direction, a rear direction, a left direction, aright direction, an upward direction, or a downward direction in thevirtual space.
 10. The system according to claim 1, wherein the objectcomprises a first object that corresponds to the user and a secondobject that does not correspond to the user, and the circuitry isconfigured to, responsive to the first position moving from the firstregion to the second region over the boundary or the first positionmoving from the first region to the boundary, move the second object inthe virtual space.
 11. A method for displaying an image of a virtualspace on a display using a computer comprising circuitry, the methodcomprising: obtaining detection data from a plurality of positiondetection sensors comprising a first position detection sensor and asecond position detection sensor; detecting a first position by thefirst position detection sensor; detecting a second position by thesecond position detection sensor; calculating a distance between thefirst position and the second position; setting, based on the distance,a boundary for determining a motion of a user, the boundary dividing afirst region from a second region; determining, when the motion of theuser occurs, whether the first position moves from the first region tothe second region over the boundary or whether the first position movesfrom the first region to the boundary; and responsive to the firstposition moving from the first region to the second region over theboundary or the first position moving from the first region to theboundary, moving an object in the virtual space.
 12. A non-transitorycomputer-readable medium that stores a program, wherein the program,when executed by a computer comprising circuitry, causes the circuitryto: obtain detection data from a plurality of position detection sensorscomprising a first position detection sensor and a second positiondetection sensor; detect a first position by the first positiondetection sensor; detect a second position by the second positiondetection sensor; calculate a distance between the first position andthe second position; set, based on the distance, a boundary fordetermining a motion of a user, the boundary dividing a first regionfrom a second region; determine, when the motion of the user occurs,whether the first position moves from the first region to the secondregion over the boundary or whether the first position moves from thefirst region to the boundary; and responsive to the first positionmoving from the first region to the second region over the boundary orthe first position moving from the first region to the boundary, move anobject in the virtual space.